StepReg: Stepwise Regression Analysis
<<<<<<< HEAD22 February 2024
Abstract
A guild on using StepReg for stepwise regression analysis with four well-known datasets namely the mtcars, remission, lung, and CreditCard for different regression models such as linear, logistic, Cox proportional hazard, and Poisson regression. The vignette explains the stepwise process with distinct parameters, offering users a clear understanding of how to effectively utilize StepReg for exploratory data analysis and model building in various regression scenarios.
StepReg is a comprehensive package aiming to simplify the stepwise regression procedures.
for simplifies the model selection procedure
A guild on using StepReg for stepwise regression analysis with four well-known datasets namely the mtcars, remission, lung, and CreditCard for different regression models such as linear, logistic, Cox proportional hazard, and Poisson regression. The vignette explains the stepwise process with distinct parameters, offering users a clear understanding of how to effectively utilize StepReg for exploratory data analysis and model building in various regression scenarios.
=======22 February 2024
Abstract
The StepReg package, developed for exploratory model building tasks, offers support across diverse scenarios. It facilitates model construction for various response variable types, including continuous (linear regression), binary (logit regression), and time-to-event (Cox regression), among others. StepReg encompasses all commonly used model selection strategies, including forward selection, backward elimination, bidirectional elimination, and best subsets. Notably, it offers flexibility in selection metrics, accommodating both information criteria (AIC, BIC, etc.) and significance level cutoffs. This vignettes provide four examples showcasing the effective utilization of StepReg for model development in diverse contexts. Furthermore, it delves into considerations for selecting appropriate strategies and metrics, empowering users to make informed decisions throughout the modeling process.
>>>>>>> 0ac6c3a (update vignettes abstract and introduction)Package
StepReg 1.5.0
1 Introduction
Model selection is the process of choosing the most relevant features from a set of candidate variables. This procedure is crucial because it ensures that the final model is both accurate and interpretable while being computationally efficient and avoiding overfitting. Stepwise regression algorithms iteratively add or remove features from the model based on certain criteria (e.g., significance level or P-value, information criteria like AIC or BIC, etc.). The process continues until no further improvements can be made according to the chosen criterion. At the end of the stepwise procedure, you’ll have a final model that includes the selected features and their coefficients.
StepReg simplifies model selection tasks by providing a unified programming interface. It currently supports model buildings for five distinct response variable types (section 3.1), four model selection strategies (section 3.2) including the best subsets algorithm, and a variety of selection metrics (section 3.3). Moreover, StepReg detects and addresses the multicollinearity issues if they exist (section 3.4). The output of StepReg includes multiple tables summarizing the final model and the variable selection procedures. Additionally, StepReg offers a plot function to visualize the selection steps (section 4). For demonstration, the vignettes include four use cases covering distinct regression scenarios (section 5).
2 Quick demo
The following example selects an optimal linear regression model with the mtcars dataset.
library(StepReg)
devtools::load_all("~/GitHub/StepReg/")
data(mtcars)
formula <- mpg ~ .
res <- stepwise(formula = formula,
data = mtcars,
type = "linear",
include = c("qsec"),
strategy = "bidirection",
metric = c("AIC"))Breakdown of the parameters:
formula: specifies the dependent and independent variablestype: specifies the regression category, depending on your data, choose from “linear”, “logit”, “cox”, etc.include: specifies the variables that must be in the final modelstrategy: specifies the model selection strategy, choose from “forward”, “backward”, “bidirection”, “subset”metric: specifies the model fit evaluation metric, choose one or more from “AIC”, “AICc”, “BIC”, “SL”, etc.
The output consists of multiple tables, which can be viewed with:
res## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## —————————————————————————————————————————————
## included variable qsec
## strategy bidirection
## metric AIC
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## intercept 1
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## ——————————————————————————————————————————————
## Dependent mpg numeric
## Independent cyl numeric
## Independent disp numeric
## Independent hp numeric
## Independent drat numeric
## Independent wt numeric
## Independent qsec numeric
## Independent vs numeric
## Independent am numeric
## Independent gear numeric
## Independent carb numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under bidirection with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn AIC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 149.943449990894
## 0 qsec 1 2 145.776053727195
<<<<<<< HEAD
## 1 wt 3 3 97.9084298151279
## 2 am 4 4 95.3073047438021
=======
## 1 wt 2 3 97.908429815128
## 2 am 3 4 95.3073047438022
>>>>>>> 0ac6c3a (update vignettes abstract and introduction)
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Parameter Estimates for mpg under bidirection with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 9.61778051456158 6.95959298481825 1.38194583153669 0.177915165458584
## qsec 1.2258859715837 0.288669553911072 4.24667567110782 0.000216173705201939
## wt -3.91650372494249 0.711201634662253 -5.50688234399576 6.95271111117161e-06
## am 2.93583719188942 1.41090451492715 2.08081919139723 0.0467155099194558
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗You can also visualize the variable selection procedures with:
plot(res)## $bidirection
The (+)1 refers to original model with intercept being added, (+) indicates variables being added to the model while (-) means variables being removed from the model.
The (+)1 refers to original model with intercept being added, (+) indicates variables being added to the model while (-) means variables being removed from the model.
Additionally, you can generate reports of various formats with:
report(res, report_name = "path_to/demo_res", format = "html")Replace "path_to/demo_res" with desired output file name, the suffix ".html" will added automatically. Supported format includes “html”, “pdf”, “docx”, etc. For detailed examples and more usage, refer to section 4 and 5.
3 Key features
3.1 Regression categories
<<<<<<< HEADStepReg supports multiple types of regressions, including linear, logit, cox, poisson, and gamma regressions. These methods primarily vary by the type of response variable (refer to the table below). Additional regression techniques can be incorporated upon user request.
=======StepReg supports multiple types of regressions, including linear, logit, cox, poisson, and gamma regressions. These methods primarily vary by the type of response variable, which are summarized in the table below. Additional regression techniques can be incorporated upon user requests.
>>>>>>> 0ac6c3a (update vignettes abstract and introduction)| Regression | Reponse |
|---|---|
| linear | continuous |
| logit | binary |
| cox | time-to-event |
| poisson | count |
| gamma | gamma distribution |
3.2 Model selection strategies
Model selection aims to identify the subset of independent variables that provide the best predictive performance for the response variable. Both stepwise regression and best subsets approaches are implemented in StepReg. For stepwise regression, there are mainly three methods: Forward Selection, Backward Elimination, Bidirectional Elimination.
| Strategy | Description |
|---|---|
| Forward Selection | In forward selection, the algorithm starts with an empty model (no predictors) and adds in variables one by one. Each step tests the addition of every possible predictor by calculating a pre-selected metric. Add the variable (if any) whose inclusion leads to the most statistically significant fit improvement. Repeat this process until more predictors no longer lead to a statistically better fit. |
| Backward Elimination | In backward elimination, the algorithm starts with a full model (all predictors) and deletes variables one by one. Each step test the deletion of every possible predictor by calculating a pre-selected metric. Delete the variable (if any) whose loss leads to the most statistically significant fit improvement. Repeat this process until less predictors no longer lead to a statistically better fit. |
| Bidirectional Elimination | Bidirectional elimination is essentially a forward selection procedure combined with backward elimination at each iteration. Each iteration starts with a forward selection step that adds in predictors, followed by a round of backward elimination that removes predictors. Repeat this process until no more predictors are added or excluded. |
| Best Subsets | Stepwise algorithms add or delete one predictor at a time and output a single model without evaluating all candidates. Therefore, it is a relatively simple procedure that only produces one model. In contrast, the Best Subsets algorithm calculates all possible models and output the best-fitting models with one predictor, two predictors, etc., for users to choose from. |
Given the computational constraints, when dealing with datasets featuring a substantial number of predictor variables greater than the sample size, the Bidirectional Elimination typically emerges as the most advisable approach. Forward Selection and Backward Elimination can be considered in sequence. On the contrary, the Best Subsets approach requires the most substantial processing time, yet it calculates a comprehensive set of models with varying numbers of variables. In practice, users can experiment with various methods and select a final model based on the specific dataset and research objectives at hand.
3.3 Selection metrics
<<<<<<< HEADVarious selection metrics can be used to guide the process of adding or removing predictors from the model. These metrics help to determine the importance or significance of predictors in improving the model fit. In StepReg, selection metrics include two categories: Information Criteria and Significance Level of the coefficient associated with each predictor. Information Criteria is a means of evaluating a model’s performance, which balances model fit with complexity by penalizing models with a higher number of parameters. Lower Information Criteria values indicate a better trade-off between model fit and complexity. Note that when evaluating different models, it’s important to compare them within the same Information Criteria framework rather than across multiple Information Criteria. For example, if you decide to use AIC, you should compare all models using AIC. This ensures consistency and fairness in model comparison, as each Information Criterion has its own scale and penalization factors. In practice, multiple metrics have been proposed, the ones supported by StepReg are summarized below.
=======Various selection metrics can be used to guide the process of adding or removing predictors from the model. These metrics help to determine the importance or significance of predictors in improving the model fit. In StepReg, selection metrics include two categories: Information Criteria and Significance Level of the coefficient associated with each predictor. Information Criteria is a means of evaluating a model’s performance, which balances model fit with complexity by penalizing models with a higher number of parameters. Lower Information Criteria values indicate a better trade-off between model fit and complexity. Note that when evaluating different models, it is important to compare them within the same Information Criteria framework rather than across multiple Information Criteria. For example, if you decide to use AIC, you should compare all models using AIC. This ensures consistency and fairness in model comparison, as each Information Criterion has its own scale and penalization factors. In practice, multiple metrics have been proposed, the ones supported by StepReg are summarized below.
>>>>>>> 0ac6c3a (update vignettes abstract and introduction)Importantly, given the discrepancies in terms of the precise definitions of each metric, StepReg mirrors the formulas adopted by SAS for univariate multiple regression (UMR) except for HQ, IC(1), and IC(3/2). A subset of the UMR can be easily extended to multivariate multiple regression (MMR), which are indicated in the following table.
| Statistic | Meanings |
|---|---|
| \({n}\) | Sample Size |
| \({p}\) | Number of parameters including the intercept |
| \({q}\) | Number of dependent variables |
| \(\sigma^2\) | Estimate of pure error variance from fitting the full model |
| \({SST}\) | Total sum of squares corrected for the mean for the dependent variable, which is a numeric value for UMR and a matrix for multivariate regression |
| \({SSE}\) | Error sum of squares, which is a numeric value for UMR and a matrix for multivariate regression |
| \(\text{LL}\) | The natural logarithm of likelihood |
| \({| |}\) | The determinant function |
| \(\ln()\) | The natural logarithm |
| Abbreviation | Definition | Formula | |
|---|---|---|---|
| linear | logit, cox, poisson and gamma | ||
| AIC | Akaike’s Information Criterion |
\(n\ln\left(\frac{|\text{SSE}|}{n}\right) + 2pq + n + q(q+1)\) (Clifford M. Hurvich 1989; Al-Subaihi 2002)\(^1\) |
\(-2\text{LL} + 2p\) (Darlington 1968; George G. Judge 1985) |
| AICc | Corrected Akaike’s Information Criterion |
\(n\ln\left(\frac{|\text{SSE}|}{n}\right) + \frac{nq(n+p)}{n-p-q-1}\) (Clifford M. Hurvich 1989; Edward J. Bedrick 1994)\(^2\) |
\(-2\text{LL} + \frac{n(n+p)}{n-p-2}\) (Clifford M. Hurvich 1989) |
| BIC | Sawa Bayesian Information Criterion |
\(n\ln\left(\frac{SSE}{n}\right) + 2(p+2)o - 2o^2, o = \frac{n\sigma^2}{SSE}\) (Sawa 1978; George G. Judge 1985) not available for MMR |
not available |
| Cp | Mallows’ Cp statistic |
\(\frac{SSE}{\sigma^2} + 2p - n\) (Mallows 1973; Hocking 1976) not available for MMR |
not available |
| HQ | Hannan and Quinn Information Criterion |
\(n\ln\left(\frac{|\text{SSE}|}{n}\right) + 2pq\ln(\ln(n))\) (E. J. Hannan 1979; Allan D R McQuarrie 1998; Clifford M. Hurvich 1989) |
\(-2\text{LL} + 2p\ln(\ln(n))\) (E. J. Hannan 1979) |
| IC(1) | Information Criterion with Penalty Coefficient Set to 1 |
\(n\ln\left(\frac{|\text{SSE}|}{n}\right) + p\) (J. A. Nelder 1972; A. F. M. Smith 1980) not available for MMR |
\(-2\text{LL} + p\) (J. A. Nelder 1972; A. F. M. Smith 1980) |
| IC(3/2) | Information Criterion with Penalty Coefficient Set to 3/2 |
\(n\ln\left(\frac{|\text{SSE}|}{n}\right) + \frac{3}{2}p\) (A. F. M. Smith 1980) not available for MMR |
\(-2\text{LL} + \frac{3}{2}p\) (A. F. M. Smith 1980) |
| SBC | Schwarz Bayesian Information Criterion |
\(n\ln\left(\frac{|\text{SSE}|}{n}\right) + p \ln(n)\) (Clifford M. Hurvich 1989; Schwarz 1978; George G. Judge 1985; Al-Subaihi 2002) not available for MMR |
\(-2\text{LL} + p\ln(n)\) (Schwarz 1978; George G. Judge 1985) |
| SL | Significance Level (pvalue) | \(\textit{F test}\) for UMR and \(\textit{Approximate F test}\) for MMR |
Forward: LRT and Rao Chi-square test (logit, poisson, gamma); LRT (cox); Backward: Wald test |
| Rsq | R-square statistic |
\(1 - \frac{SSE}{SST}\) not available for MMR |
not available |
| adjRsq | Adjusted R-square statistic |
\(1 - \frac{(n-1)(1-R^2)}{n-p}\) (Darlington 1968; George G. Judge 1985) not available for MMR |
not available |
|
1 Unsupported AIC formula (which does not affect the selection process as it only differs by constant additive and multiplicative factors): \(AIC=n\ln\left(\frac{SSE}{n}\right) + 2p\) (Darlington 1968; George G. Judge 1985) |
|||
|
2 Unsupported AICc formula (which does not affect the selection process as it only differs by constant additive and multiplicative factors): \(AICc=\ln\left(\frac{SSE}{n}\right) + 1 + \frac{2(p+1)}{n-p-2}\) (Allan D R McQuarrie 1998) |
No metric is necessarily optimal for all datasets. The choice of them depends on your data and research goals. We recommend using multiple metrics simultaneously, which allows the selection of the best model based on your specific needs. Below summarizes general guidance.
-
<<<<<<< HEAD
AIC: AIC works by penalizing the inclusion of additional variables in a model. The lower the AIC, the better performance of the model. AIC does not include sample size in penalty calculation, and it is optimal in minimizing the mean square error of predictions (Mark J. Brewer 2016).
AICc: AICc is a variant of AIC, which works better for small sample size, especially when
numObs / numParam < 40(Kenneth P. Burnham 2002).Cp: Cp is used for linear models. It is equivalent to AIC when dealing with Gaussian linear model selection.
IC(1) and IC(3/2): IC(1) and IC(3/2) have 1 and 3/2 as penalty factors respectively, compared to 2 used by AIC. As such, IC(1) turns to return a complex model with more variables that may suffer from overfitting issues.
BIC and SBC: Both BIC and SBC are variants of Bayesian Information Criterion. The main distinction between BIC/SBC and AIC lies in the magnitude of the penalty imposed: BIC/SBC are more parsimonious when penalizing model complexity, which typically results to a simpler model (SAS Institute Inc 2018; Sawa 1978; Clifford M. Hurvich 1989; Schwarz 1978; George G. Judge 1985; Al-Subaihi 2002).
The precise definitions of these criteria can vary across literature and in the SAS environment. Here, BIC aligns with the definition of the Sawa Bayesion Information Criterion as outlined in SAS documentation, while SBC corresponds to the Schwarz Bayesian Information Criterion. According to Richard’s post, whereas AIC often favors selecting overly complex models, BIC/SBC prioritize a small models. Consequently, when dealing with a limited sample size, AIC may seem preferable, whereas BIC/SBC tend to perform better with larger sample sizes.
HQ: HQ is an alternative to AIC, differing primarily in the method of penalty calculation. However, HQ has remained relatively underutilized in practice (Kenneth P. Burnham 2002).
Rsq: The R-squared (R²) statistic measures the proportion of variations that is explained by the model. It ranges from 0 to 1, with 1 indicating that all of the variability in the response variables is accounted for by the independent variables. As such, R-squared is valuable for communicating the explanatory power of a model. However, R-squared alone is not sufficient for selection because it does not take into account the complexity of the model. Therefore, while R-squared is useful for understanding how well the model fits the data, it should not be the sole criterion for model selection.
adjRsq: The adjusted R-squared (adj-R²) seeks to overcome the limitation of R-squared in model selection by considering the number of predictors. It serves a similar purpose to information criteria, as both methods compare models by weighing their goodness of fit against the number of parameters. However, information criteria are typically regarded as superior in this context (Stevens 2016).
=======
AIC: AIC works by penalizing the inclusion of additional variables in a model. The lower the AIC, the better performance of the model. AIC does not include sample size in penalty calculation, and it is optimal in minimizing the mean square error of predictions (Mark J. Brewer 2016).
AICc: AICc is a variant of AIC, which works better for small sample size, especially when
numObs / numParam < 40(Kenneth P. Burnham 2002).Cp: Cp is used for linear models. It is equivalent to AIC when dealing with Gaussian linear model selection.
IC(1) and IC(3/2): IC(1) and IC(3/2) have 1 and 3/2 as penalty factors respectively, compared to 2 used by AIC. As such, IC(1) turns to return a complex model with more variables that may suffer from overfitting issues.
BIC and SBC: Both BIC and SBC are variants of Bayesian Information Criterion. The main distinction between BIC/SBC and AIC lies in the magnitude of the penalty imposed: BIC/SBC are more parsimonious when penalizing model complexity, which typically results to a simpler model (Inc 2018; Sawa 1978; Clifford M. Hurvich 1989; Schwarz 1978; George G. Judge 1985; Al-Subaihi 2002).
The precise definitions of these criteria can vary across literature and in the SAS environment. Here, BIC aligns with the definition of the Sawa Bayesion Information Criterion as outlined in SAS documentation, while SBC corresponds to the Schwarz Bayesian Information Criterion. According to Richard’s post, whereas AIC often favors selecting overly complex models, BIC/SBC prioritize a small models. Consequently, when dealing with a limited sample size, AIC may seem preferable, whereas BIC/SBC tend to perform better with larger sample sizes.
HQ: HQ is an alternative to AIC, differing primarily in the method of penalty calculation. However, HQ has remained relatively underutilized in practice (Kenneth P. Burnham 2002).
Rsq: The R-squared (R²) statistic measures the proportion of variations that is explained by the model. It ranges from 0 to 1, with 1 indicating that all of the variability in the response variables is accounted for by the independent variables. As such, R-squared is valuable for communicating the explanatory power of a model. However, R-squared alone is not sufficient for selection because it does not take into account the complexity of the model. Therefore, while R-squared is useful for understanding how well the model fits the data, it should not be the sole criterion for model selection.
adjRsq: The adjusted R-squared (adj-R²) seeks to overcome the limitation of R-squared in model selection by considering the number of predictors. It serves a similar purpose to information criteria, as both methods compare models by weighing their goodness of fit against the number of parameters. However, information criteria are typically regarded as superior in this context (Stevens 2016).
>>>>>>> 0ac6c3a (update vignettes abstract and introduction)
SL: SL stands for Significance Level (P-value), embodying a distinct approach to model selection in contrast to information criteria. The SL method operates by calculating a P-value through specific hypothesis testing. Should this P-value fall below a predefined threshold, such as 0.05, one should favor the alternative hypothesis, indicating that the full model significantly outperforms the reduced model. The effectiveness of this method hinges upon the selection of the P-value threshold, wherein smaller thresholds tend to yield simpler models.
3.4 Multicollinearity
This blog by Jim Frost gives an excellent overview of multicollinearity and when it is necessary to remove it.
Simply put, a dataset contains multicollinearity when input predictors are correlated. When multicollinearity occurs, the interpretability of predictors will be badly affected because changes in one input variable lead to changes in other input variables. Therefore, it is hard to individually estimate the relationship between each input variable and the dependent variable.
Multicollinearity can dramatically reduce the precision of the estimated regression coefficients of correlated input variables, making it hard to find the correct model. However, as Jim pointed out, “Multicollinearity affects the coefficients and p-values, but it does not influence the predictions, precision of the predictions, and the goodness-of-fit statistics. If your primary goal is to make predictions, and you don’t need to understand the role of each independent variable, you don’t need to reduce severe multicollinearity.”
In StepReg, QC Matrix Decomposition is performed ahead of time to detect and remove input variables causing multicollinearity.
4 StepReg output
A list containing multiple tables will be returned.
| Table | Meanings |
|---|---|
| Summary of Parameters | The parameters used in stepwise regression along with their default or user-specified values |
| Variables and Types | The variables and their respective types in the dataset |
| Selection Process under a given strategy with metrics | The detailed overview of the variable selection process under a given strategy with metrics |
| Parameter Estimates for y under a given strategy with metrics | The parameter estimates for the optimal models under a given strategy with metrics |
5 Use cases
In this section, we provided some examples using distinct parameters across various regression scenarios with the 4 datasets, a.k.a. mtcars, remission, lung, and CreditCard.
5.1 linear stepwise regression with mtcars
- mtcars: the mtcars dataset is a classic automotive dataset that provides information on various car models and their performance attributes. With 32 observations and 11 variables, it includes details such as miles per gallon (mpg), horsepower(hp), and the number of cylinders(cyl). For more information, please
?mtcars
Example1:
type of regression:
linearresponse:
mpgpredictors: all variables except
mpgvariable selection strategy:
forwardandbackwardselection metric:
AICforce
dispandcylto be included in all models.
library(StepReg)
data(mtcars)
formula <- mpg ~ .
res1 <- stepwise(formula = formula,
data = mtcars,
type = "linear",
include = c("disp","cyl"),
strategy = c("forward","backward"),
metric = "AIC")
res1## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## ————————————————————————————————————————————————————
## included variable disp cyl
## strategy forward & backward
## metric AIC
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## intercept 1
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## ——————————————————————————————————————————————
## Dependent mpg numeric
## Independent cyl numeric
## Independent disp numeric
## Independent hp numeric
## Independent drat numeric
## Independent wt numeric
## Independent qsec numeric
## Independent vs numeric
## Independent am numeric
## Independent gear numeric
## Independent carb numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under forward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn AIC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 149.943449990894
## 0 disp cyl 2 3 108.33357089067
## 1 wt 4 4 98.7462938182664
## 2 hp 5 5 97.5255371708581
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Parameter Estimates for mpg under forward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ——————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 40.8285367422432 2.75746792810596 14.8065318642844 1.76140221350856e-14
## disp 0.0115992393009777 0.0117268091002486 0.989121525030347 0.331385561864358
## cyl -1.29331972351378 0.655876754872712 -1.97189443581482 0.0589468066844993
## wt -3.85390352303832 1.01547364107822 -3.79517829625422 0.000758947039357617
## hp -0.020538376368824 0.0121467704321512 -1.69085078898512 0.102379131471602
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 5. Selection Process under backward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn AIC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 11 11 104.897744309381
## 1 vs 10 10 102.932465913469
## 2 carb 9 9 101.031851946668
## 3 gear 8 8 99.2420251061788
## 4 drat 7 7 97.6475225735095
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 6. Parameter Estimates for mpg under backward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ———————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 20.0516995179774 13.3048610392912 1.50709574934768 0.144319410164763
## disp 0.0139609925549473 0.0115486434649119 1.20888592650426 0.238016363134663
## cyl -0.502065773676562 0.788817530923406 -0.636478975167848 0.530249264850094
## hp -0.0195605432188204 0.0148899686558553 -1.31367255841257 0.200884933942454
## wt -3.99773179978435 1.21563794662227 -3.28858753619226 0.00298829668548213
## qsec 0.810177821325826 0.571713217165793 1.41710528460789 0.16879463401253
## am 2.94074955171802 1.71809777458412 1.71163108131599 0.0993453133035962
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗Visulization of the selection process under
AIC.
plot(res1)## $forward
##
## $backward
##
## $backward
Example2:
type of regression:
linearresponse:
mpgpredictors: all other variables except
mpgandintercept.variable selection strategy:
bidirectionalselection metric: run
AIC,AICc,BIC,HQ,HQc,SBC, andSLparallelly, and the significance levels for entry (sle) and stay (sls) were both set to 0.05 .Users can compare output within each metic through the output table and visualization.
formula <- mpg ~ . + 0
res2 <- stepwise(formula = formula,
data = mtcars,
type = "linear",
strategy = "bidirection",
metric = c("AIC","SBC","SL","AICc","BIC","HQ"),
sle = 0.05,
sls = 0.05)
res2## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## ——————————————————————————————————————————————————————————————————
## included variable NULL
## strategy bidirection
## metric AIC & SBC & SL & AICc & BIC & HQ
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## intercept 0
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## ——————————————————————————————————————————————
## Dependent mpg numeric
## Independent cyl numeric
## Independent disp numeric
## Independent hp numeric
## Independent drat numeric
## Independent wt numeric
## Independent qsec numeric
## Independent vs numeric
## Independent am numeric
## Independent gear numeric
## Independent carb numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under bidirection with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn AIC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 0 0 0 Inf
## 1 drat 1 1 131.940615397799
## 2 carb 2 2 112.874977003721
## 3 gear 3 3 105.767914676203
## 4 hp 4 4 105.399654906399
## 5 qsec 5 5 105.277861812131
## 6 wt 6 6 100.437613232709
## 7 hp 5 5 98.4440186423183
## 8 am 6 6 97.8009315992234
## 9 gear 5 5 96.5751826224886
## 10 carb 4 4 96.0485980614552
## 11 drat 3 3 95.4186908507389
## 12 disp 4 4 95.3954043177414
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Selection Process under bidirection with SBC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn SBC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 0 0 0 Inf
## 1 drat 1 1 99.4063513005992
## 2 carb 2 2 81.8064488093206
## 3 gear 3 3 76.1651223846019
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 5. Selection Process under bidirection with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn SL
## —————————————————————————————————————————————————————————————————————————————————————————
## 0 0 0 0 1
## 1 drat 1 1 2.44913223058495e-22
## 2 carb 2 2 1.03775546495334e-05
## 3 gear 3 3 0.00438861027007104
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 6. Selection Process under bidirection with AICc
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn AICc
## —————————————————————————————————————————————————————————————————————————————————————
## 0 0 0 0 Inf
## 1 drat 1 1 132.354408501248
## 2 carb 2 2 113.732119860864
## 3 gear 3 3 107.249396157684
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 7. Selection Process under bidirection with BIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn BIC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 0 0 0 Inf
## 1 drat 1 1 97.7553842586695
## 2 carb 2 2 79.2794817092699
## 3 gear 3 3 72.9946131788243
## 4 hp 4 4 72.9415244192211
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 8. Selection Process under bidirection with HQ
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn HQ
## —————————————————————————————————————————————————————————————————————————————————————
## 0 0 0 0 Inf
## 1 drat 1 1 98.4264653815043
## 2 carb 2 2 79.8466769711309
## 3 gear 3 3 73.2254646273174
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 9. Parameter Estimates for mpg under bidirection with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ———————————————————————————————————————————————————————————————————————————————————————————
## qsec 1.70550996283541 0.127485704584404 13.3780486870687 1.09964868080962e-13
## wt -4.61279456246674 1.15817323630342 -3.98281916545536 0.000440008628764362
## am 4.18085430467978 1.01361607335742 4.1246921932004 0.000300527233592528
## disp 0.0120200576653963 0.0088914542638529 1.35186633240215 0.187238258962162
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 10. Parameter Estimates for mpg under bidirection with SBC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## —————————————————————————————————————————————————————————————————————————————————————————
## drat 3.85142334610757 1.0678868653112 3.6065836852345 0.00115059878350687
## carb -2.36055514388328 0.350142761922923 -6.74169339077443 2.12967212881875e-07
## gear 3.48835425113011 1.12895206584177 3.08990466174409 0.00438861027007103
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 11. Parameter Estimates for mpg under bidirection with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## —————————————————————————————————————————————————————————————————————————————————————————
## drat 3.85142334610757 1.0678868653112 3.6065836852345 0.00115059878350687
## carb -2.36055514388328 0.350142761922923 -6.74169339077443 2.12967212881875e-07
## gear 3.48835425113011 1.12895206584177 3.08990466174409 0.00438861027007103
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 12. Parameter Estimates for mpg under bidirection with AICc
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## —————————————————————————————————————————————————————————————————————————————————————————
## drat 3.85142334610757 1.0678868653112 3.6065836852345 0.00115059878350687
## carb -2.36055514388328 0.350142761922923 -6.74169339077443 2.12967212881875e-07
## gear 3.48835425113011 1.12895206584177 3.08990466174409 0.00438861027007103
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 13. Parameter Estimates for mpg under bidirection with BIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ————————————————————————————————————————————————————————————————————————————————————————————
## drat 4.30273288409265 1.09158280873171 3.94173749318378 0.000491150652824569
## carb -1.73804836219828 0.54597600366577 -3.18337866596471 0.00355127557199681
## gear 3.17367474781417 1.12779616721381 2.81404994987228 0.00885014913869605
## hp -0.0155479574747606 0.0106015597450293 -1.46657264107297 0.153634601937482
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 14. Parameter Estimates for mpg under bidirection with HQ
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## —————————————————————————————————————————————————————————————————————————————————————————
## drat 3.85142334610757 1.0678868653112 3.6065836852345 0.00115059878350687
## carb -2.36055514388328 0.350142761922923 -6.74169339077443 2.12967212881875e-07
## gear 3.48835425113011 1.12895206584177 3.08990466174409 0.00438861027007103
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
- Visulization of the selection process using
bidirectionstrategy under information criteronAIC,AICc,BIC,HQ,SBC, andSLwithsle=0.05 andsls=0.05.
plot(res2)## $bidirection
Example3:
type of regression:
linearresponse: multivariates of
mpganddratpredictors:
cyl,disp,hp,wt,vs,amandintercept.variable selection strategy:
subsetselection metric: run
AIC,SBCandHQparallelly
formula <- cbind(mpg,drat) ~ cyl + disp + hp + wt + vs + am
res3 <- stepwise(formula = formula,
data = mtcars,
type = "linear",
include = 'wt',
strategy = "subset",
metric = c("AIC","AICc","SBC","HQ"),
best_n = 3)
res3## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## ———————————————————————————————————————————————————————
## included variable wt
## strategy subset
## metric AIC & AICc & SBC & HQ
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## intercept 1
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## —————————————————————————————————————————————————
## Dependent cbind(mpg, drat) nmatrix.2
## Independent cyl numeric
## Independent disp numeric
## Independent hp numeric
## Independent wt numeric
## Independent vs numeric
## Independent am numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under subset with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## NumberOfVariables AIC VariablesInModel
## —————————————————————————————————————————————————————————————
## 2 161.304784331389 1 wt
## 3 150.7504611172 1 wt cyl
## 3 153.019337447204 1 wt hp
## 3 158.369641697526 1 wt vs
## 4 146.595187612804 1 wt cyl am
## 4 147.017463011528 1 wt cyl hp
## 4 148.411791388542 1 wt hp am
## 5 145.725338025984 1 wt cyl hp am
## 5 148.951593685218 1 wt hp vs am
## 5 149.437618782264 1 wt cyl disp hp
## 6 148.135750495875 1 wt cyl disp hp am
## 6 149.129861532524 1 wt cyl hp vs am
## 6 150.642710344858 1 wt disp hp vs am
## 7 151.290651094496 1 wt cyl disp hp vs am
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Selection Process under subset with AICc
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## NumberOfVariables AICc VariablesInModel
## —————————————————————————————————————————————————————————————
## 2 195.897376923981 1 wt
## 3 186.904307271046 1 wt cyl
## 3 189.17318360105 1 wt hp
## 3 194.523487851373 1 wt vs
## 4 184.755187612804 1 wt cyl am
## 4 185.177463011528 1 wt cyl hp
## 4 186.571791388542 1 wt hp am
## 5 186.392004692651 1 wt cyl hp am
## 5 189.618260351885 1 wt hp vs am
## 5 190.104285448931 1 wt cyl disp hp
## 6 191.874880930657 1 wt cyl disp hp am
## 6 192.868991967306 1 wt cyl hp vs am
## 6 194.381840779641 1 wt disp hp vs am
## 7 198.745196549042 1 wt cyl disp hp vs am
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 5. Selection Process under subset with SBC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## NumberOfVariables SBC VariablesInModel
## —————————————————————————————————————————————————————————————
## 2 129.167727942588 1 wt
## 3 121.544876533998 1 wt cyl
## 3 123.813752864002 1 wt hp
## 3 129.164057114325 1 wt vs
## 4 120.321074835202 1 wt cyl am
## 4 120.743350233926 1 wt cyl hp
## 4 122.137678610939 1 wt hp am
## 5 122.382697053981 1 wt cyl hp am
## 5 125.608952713216 1 wt hp vs am
## 5 126.094977810261 1 wt cyl disp hp
## 6 127.724581329471 1 wt cyl disp hp am
## 6 128.71869236612 1 wt cyl hp vs am
## 6 130.231541178455 1 wt disp hp vs am
## 7 133.810953733693 1 wt cyl disp hp vs am
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 6. Selection Process under subset with HQ
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## NumberOfVariables HQ VariablesInModel
## —————————————————————————————————————————————————————————————
## 2 125.248184266208 1 wt
## 3 115.665561019429 1 wt cyl
## 3 117.934437349433 1 wt hp
## 3 123.284741599756 1 wt vs
## 4 112.481987482443 1 wt cyl am
## 4 112.904262881167 1 wt cyl hp
## 4 114.298591258181 1 wt hp am
## 5 112.583837863033 1 wt cyl hp am
## 5 115.810093522267 1 wt hp vs am
## 5 116.296118619313 1 wt cyl disp hp
## 6 115.965950300333 1 wt cyl disp hp am
## 6 116.960061336982 1 wt cyl hp vs am
## 6 118.472910149317 1 wt disp hp vs am
## 7 120.092550866365 1 wt cyl disp hp vs am
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 7. Parameter Estimates for Response mpg under subset with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ——————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 36.1465357519024 3.10478079459192 11.6422182895696 4.9448037493366e-12
## wt -2.60648070821659 0.919837490381643 -2.83363173981434 0.00860321812827095
## cyl -0.74515702393006 0.582787409987314 -1.27860865070212 0.211916611111084
## hp -0.0249510591437429 0.0136461447865208 -1.82843283096253 0.0785533736998691
## am 1.47804770540897 1.44114927311401 1.02560347701889 0.31417988631753
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 8. Parameter Estimates for Response drat under subset with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ————————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 4.53925999228833 0.40732366563224 11.1441106306519 1.32299213312142e-11
## wt -0.0936785284603612 0.120675694406779 -0.77628331803574 0.444329239934456
## cyl -0.1691236910122 0.0764572830822192 -2.21200236516815 0.0356145054513228
## hp 0.00171645750563224 0.00179027058073661 0.958769877638275 0.346182118711364
## am 0.377500876754036 0.189067841977936 1.99664243694118 0.0560372760709089
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 9. Parameter Estimates for Response mpg under subset with AICc
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 39.4179334351865 2.6414572997099 14.9227978962656 7.42499755293903e-15
## wt -3.12514220026708 0.910882701148664 -3.43089422636541 0.00188589438685629
## cyl -1.5102456624971 0.422279222208057 -3.57641480582487 0.00129160458914754
## am 0.176493157719671 1.30445145498685 0.135300671439283 0.893342147923959
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 10. Parameter Estimates for Response drat under subset with AICc
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ———————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 4.31421082403332 0.332409660876704 12.9785963881282 2.29282220122696e-13
## wt -0.0579982631143611 0.114628470360106 -0.505967347659435 0.616840130742602
## cyl -0.116490969949067 0.053141004045333 -2.19211081991774 0.0368483162012922
## am 0.467038681922973 0.164156454783472 2.84508265324694 0.00821005565960179
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 11. Parameter Estimates for Response mpg under subset with SBC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 39.4179334351865 2.6414572997099 14.9227978962656 7.42499755293903e-15
## wt -3.12514220026708 0.910882701148664 -3.43089422636541 0.00188589438685629
## cyl -1.5102456624971 0.422279222208057 -3.57641480582487 0.00129160458914754
## am 0.176493157719671 1.30445145498685 0.135300671439283 0.893342147923959
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 12. Parameter Estimates for Response drat under subset with SBC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ———————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 4.31421082403332 0.332409660876704 12.9785963881282 2.29282220122696e-13
## wt -0.0579982631143611 0.114628470360106 -0.505967347659435 0.616840130742602
## cyl -0.116490969949067 0.053141004045333 -2.19211081991774 0.0368483162012922
## am 0.467038681922973 0.164156454783472 2.84508265324694 0.00821005565960179
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 13. Parameter Estimates for Response mpg under subset with HQ
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 39.4179334351865 2.6414572997099 14.9227978962656 7.42499755293903e-15
## wt -3.12514220026708 0.910882701148664 -3.43089422636541 0.00188589438685629
## cyl -1.5102456624971 0.422279222208057 -3.57641480582487 0.00129160458914754
## am 0.176493157719671 1.30445145498685 0.135300671439283 0.893342147923959
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 14. Parameter Estimates for Response drat under subset with HQ
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error t value Pr(>|t|)
## ———————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 4.31421082403332 0.332409660876704 12.9785963881282 2.29282220122696e-13
## wt -0.0579982631143611 0.114628470360106 -0.505967347659435 0.616840130742602
## cyl -0.116490969949067 0.053141004045333 -2.19211081991774 0.0368483162012922
## am 0.467038681922973 0.164156454783472 2.84508265324694 0.00821005565960179
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
- Visulization of the selection process using
subsetstrategy under information criteronAIC,SBCandHQ.
plot(res3)## $subset
5.2 Logistic stepwise regression with remission
- remission: the remission dataset is relevant in the context of medical research, specifically in oncology. It captures data related to the remission status of leukemia patients. With 27 observations and 7 variables, the dataset includes variables such as remission status (1 for remission and 0 for non-remission), cellularity of the marrow clot section(cell), and the highest temperature before the start of treatment(temp). For more information, please
?StepReg::remission
Example4:
type of regression:
logitresponse:
remisspredictors: all variables except
remissvariable selection strategy:
forwardselection metric: run
AICandSLparallelly, wheresleandslswere both set to 0.05.force
cellalways in the model.
data(remission)
formula <- remiss ~ .
res4 <- stepwise(formula = formula,
data = remission,
type = "logit",
include= "cell",
strategy = "forward",
metric = c("AIC","SL"),
sle = 0.05)
res4## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## ——————————————————————————————————————————
## included variable cell
## strategy forward
## metric AIC & SL
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## intercept 1
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## ——————————————————————————————————————————————
## Dependent remiss numeric
## Independent cell numeric
## Independent smear numeric
## Independent infil numeric
## Independent li numeric
## Independent blast numeric
## Independent temp numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under forward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn AIC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 36.3717650879199
## 0 cell 1 2 35.7917917196118
## 1 li 3 3 30.3407188099083
## 2 temp 4 4 29.9533681094191
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Selection Process under forward with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn SL
## ————————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 1
## 0 cell 1 2 0.169276579083177
## 1 li 3 3 0.00801481059473559
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 5. Parameter Estimates for remiss under forward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## ————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 67.6339061281096 56.8875473471752 1.18890529267066 0.234476937196257
## cell 9.65215222462195 7.75107586200393 1.24526612775617 0.213033942178288
## li 3.86710032908391 1.77827772175707 2.17463238827675 0.0296576752212305
## temp -82.0737742795391 61.7123821323399 -1.3299401423775 0.183537993940991
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 6. Parameter Estimates for remiss under forward with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## —————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) -9.58583007808602 6.27432627790571 -1.52778635561899 0.126565591026529
## cell 6.29163359335162 6.15249803344085 1.02261448263039 0.306490159285689
## li 2.87858063854095 1.25185701053452 2.299448430865 0.0214794885107116
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
- Visulization of the selection process using
forwardstrategy under information criteronAICandSL.
plot(res4)## $forward
Example5:
type of regression:
logitresponse:
remisspredictors: all variables except
remissvariable selection strategy:
subsetselection metric: run
SBCandSLparallelly, wheresleandslsused default 0.15.
data(remission)
formula <- remiss ~ .
res5 <- stepwise(formula = formula,
data = remission,
type = "logit",
strategy = "subset",
metric = c("SBC","SL"),
best_n = 3)
res5## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## ——————————————————————————————————————————
## included variable NULL
## strategy subset
## metric SBC & SL
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## intercept 1
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## ——————————————————————————————————————————————
## Dependent remiss numeric
## Independent cell numeric
## Independent smear numeric
## Independent infil numeric
## Independent li numeric
## Independent blast numeric
## Independent temp numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under subset with SBC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## NumberOfVariables SBC VariablesInModel
## ————————————————————————————————————————————————————————————————————————
## 2 32.664638237101 1 li
## 2 37.4121756949106 1 blast
## 2 38.3834654516204 1 cell
## 3 34.2282294079213 1 cell li
## 3 34.5353328666249 1 li temp
## 3 35.3779945497744 1 infil li
## 4 35.1367155734364 1 cell li temp
## 4 36.7176944326211 1 li blast temp
## 4 36.9592912439668 1 infil li temp
## 5 38.3371749729945 1 cell smear li temp
## 5 38.348324175947 1 cell infil li temp
## 5 38.4116928987128 1 cell li blast temp
## 6 41.5300662487924 1 cell smear infil li temp
## 6 41.6321675491241 1 cell smear li blast temp
## 6 41.6436796938359 1 cell infil li blast temp
## 7 44.8215103474761 1 cell smear infil li blast temp
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Selection Process under subset with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## NumberOfVariables SL VariablesInModel
## ————————————————————————————————————————————————————————————————————————
## 2 7.93109912391463 1 li
## 2 3.52581053761539 1 blast
## 2 1.88933825259032 1 cell
## 3 8.66108166784946 1 cell li
## 3 8.36482911322912 1 li temp
## 3 8.17463918794754 1 infil li
## 4 9.25024541927854 1 cell li temp
## 4 8.79127825261336 1 smear infil li
## 4 8.68174276768289 1 cell li blast
## 5 9.4475919767131 1 smear infil li temp
## 5 9.27906944643617 1 cell smear li temp
## 5 9.26147971975757 1 cell infil li temp
## 6 9.46088500719724 1 cell smear infil li temp
## 6 9.45015507221271 1 smear infil li blast temp
## 6 9.32952313436337 1 cell smear li blast temp
## 7 9.46088923922501 1 cell smear infil li blast temp
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 5. Parameter Estimates for remiss under subset with SBC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## ——————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) -3.77714015769881 1.37862405968586 -2.73978981518681 0.00614784861174814
## li 2.89726385662888 1.18681962433562 2.44119982280438 0.0146385524077831
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 6. Parameter Estimates for remiss under subset with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## ——————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) 58.0384871144683 71.236433417961 0.814730389068509 0.415226654399412
## cell 24.6615438508076 47.8376944382506 0.515525343359533 0.606185964565691
## smear 19.293574580837 57.9500115690827 0.332934783935927 0.739183512053813
## infil -19.601261237028 61.6814798296476 -0.317781954829277 0.750650339683557
## li 3.89596332799397 2.33711543506737 1.66699653322074 0.0955150942220505
## blast 0.151092333239225 2.27857061152582 0.0663101386785849 0.947130911494025
## temp -87.4339023538895 67.5735358529573 -1.29390746318424 0.195697386304529
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
- Visulization of the selection process using
subsetstrategy under information criteronSBCandSL.
plot(res5)## $subset
5.3 Cox stepwise regression with lung
- lung: the lung dataset is a dataset in the survival analysis domain, containing information related to the survival times of 228 patients with advanced lung cancer. It includes variables such as the patient’s age, the type of treatment received, and survival status. For more information, please
?survival::lung
Example6:
type of regression:
coxresponse:
Surv(time, status_binary)predictors: all variables except
statusvariable selection strategy:
forwardselection metric: run
IC(1)andSLparallelly, whereslewas set to 0.05.force
agein all models.
lung <- survival::lung
lung_noNA <- na.omit(lung)
lung_noNA$status_binary <- ifelse(lung_noNA$status == 2,1,0)
formula = Surv(time, status_binary) ~ . - status
res6 <- stepwise(formula = formula,
data = lung_noNA,
type = "cox",
include = "age",
strategy = "forward",
metric = c("IC(1)","SL"),
sle = 0.05)
res6## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## ————————————————————————————————————————————
## included variable age
## strategy forward
## metric IC(1) & SL
## entry significance level (sle) 0.05
## stay significance level (sls) 0.15
## test method efron
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## ——————————————————————————————————————————————————————————
## Dependent Surv(time, status_binary) nmatrix.2
## Independent inst numeric
## Independent age numeric
## Independent sex numeric
## Independent ph.ecog numeric
## Independent ph.karno numeric
## Independent pat.karno numeric
## Independent meal.cal numeric
## Independent wt.loss numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under forward with IC(1)
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn IC(1)
## —————————————————————————————————————————————————————————————————————————————————————
## 0 age 1 1 1013.71004248687
## 1 ph.ecog 2 2 1005.03577133648
## 2 sex 3 3 999.220449574499
## 3 inst 4 4 996.745082496922
## 4 ph.karno 5 5 993.164700650656
## 5 wt.loss 6 6 990.378814285337
## 6 pat.karno 7 7 989.5365169151
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Selection Process under forward with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn SL
## ————————————————————————————————————————————————————————————————————————————————————————
## 0 age 1 1 0.0605025541359114
## 1 ph.ecog 2 2 0.00186866385928123
## 2 sex 3 3 0.00903790177522826
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 5. Parameter Estimates for Surv(time, status_binary) under forward with IC(1)
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable coef exp(coef) se(coef) z Pr(>|z|)
## —————————————————————————————————————————————————————————————————————————————————————————————————————————————————
## age 0.0127911717927875 1.01287332875159 0.0117657197510269 1.08715591255444 0.276967911173407
## ph.ecog 0.907317172186787 2.47766645634186 0.238503963744317 3.80420164907388 0.000142262259259764
## sex -0.5668681013351 0.56729938352366 0.200032540541155 -2.83387942682491 0.00459866794108441
## inst -0.0303746283354971 0.970082045244345 0.0131043742343092 -2.3178999464142 0.0204547593691531
## ph.karno 0.026580081421336 1.02693648250202 0.0116170285677177 2.28802755079718 0.0221359167402799
## wt.loss -0.0167121591832758 0.983426714247836 0.00791193897857379 -2.11227099052884 0.0346632125632795
## pat.karno -0.0108962907298638 0.98916285881416 0.00799900477152 -1.36220580448451 0.173132944510343
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 6. Parameter Estimates for Surv(time, status_binary) under forward with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable coef exp(coef) se(coef) z Pr(>|z|)
## ——————————————————————————————————————————————————————————————————————————————————————————————————————————————
## age 0.00803434264043171 1.00806670458218 0.011086104693833 0.724721880436602 0.46862266905784
## ph.ecog 0.455257333389347 1.5765790372648 0.136856945048725 3.32651977016775 0.000879377814698571
## sex -0.502179683704786 0.605210054490898 0.197336202068922 -2.5447924832839 0.0109342697305065
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
- Visulization of the selection process using
forwardstrategy under information criteronIC(1)andSL.
plot(res6)## $forward
Example7:
type of regression:
coxresponse:
Surv(time, status_binary)predictors: all variables except
statusvariable selection strategy:
backwardselection metric: run
SLandAICparallelly, whereslswas set to 0.05.
formula = Surv(time, status_binary) ~ . - status
res7 <- stepwise(formula = formula,
data = lung_noNA,
type = "cox",
strategy = "backward",
metric = c("SL","AIC"),
sls = 0.05)
res7## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## ——————————————————————————————————————————
## included variable NULL
## strategy backward
## metric SL & AIC
## entry significance level (sle) 0.15
## stay significance level (sls) 0.05
## test method efron
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## ——————————————————————————————————————————————————————————
## Dependent Surv(time, status_binary) nmatrix.2
## Independent inst numeric
## Independent age numeric
## Independent sex numeric
## Independent ph.ecog numeric
## Independent ph.karno numeric
## Independent pat.karno numeric
## Independent meal.cal numeric
## Independent wt.loss numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under backward with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn SL
## ———————————————————————————————————————————————————————————————————————————————————————
## 0 8 8 1
## 1 meal.cal 7 7 0.992244442233114
## 2 age 6 6 0.276967911173407
## 3 pat.karno 5 5 0.150201287416114
## 4 ph.karno 4 4 0.0554707154521576
## 5 wt.loss 3 3 0.0652481881858785
## 6 inst 2 2 0.0670860630827791
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Selection Process under backward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn AIC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 8 8 998.536422466941
## 1 meal.cal 7 7 996.5365169151
## 2 age 6 6 995.742173541071
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 5. Parameter Estimates for Surv(time, status_binary) under backward with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable coef exp(coef) se(coef) z Pr(>|z|)
## —————————————————————————————————————————————————————————————————————————————————————————————————————————————
## sex -0.510099064468472 0.600436093983396 0.196899845516193 -2.59065243617229 0.00957941855627087
## ph.ecog 0.48251852871466 1.62014966165472 0.132315991621882 3.6467136194206 0.000265615670889065
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 6. Parameter Estimates for Surv(time, status_binary) under backward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable coef exp(coef) se(coef) z Pr(>|z|)
## —————————————————————————————————————————————————————————————————————————————————————————————————————————————————
## inst -0.0291538752858262 0.971266998980296 0.0129546401503024 -2.25045813296061 0.0244198783334591
## sex -0.562968129724542 0.569516154877045 0.199295953293519 -2.82478454991715 0.00473124175773548
## ph.ecog 0.901507779010255 2.46331444633504 0.240838552326673 3.74320377822007 0.000181688764023633
## ph.karno 0.0238044694133571 1.02409005738304 0.0113996516289851 2.08817516430336 0.0367820367652906
## pat.karno -0.0115478833807012 0.988518537505187 0.00802593552960615 -1.43882084999353 0.150201287416114
## wt.loss -0.0168103912518805 0.983330114952026 0.00781085562695218 -2.15218307119574 0.0313829384207112
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
- Visulization of the selection process using
backwardstrategy under information criteronAICandSL.
plot(res7)## $backward
5.4 Poisson stepwise regression with CreditCard
- CreditCard: the CreditCard dataset is associated with credit risk analysis and financial research. It contains information about credit card transactions, including details such as the amount spent, credit limit, and payment status. For more information, please
?AER::CreditCard
Example8:
type of regression:
poissonresponse:
reportspredictors: all variables except
reportsvariable selection strategy:
forwardselection metric: run
SLandIC(3/2)parallelly, andslewas set to 0.05.
data(CreditCard, package = 'AER')
formula = reports ~ .
res8 <- stepwise(formula = formula,
data = CreditCard,
type = "poisson",
strategy = "forward",
metric = c("SL","IC(3/2)"),
sle=0.05)
res8## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## ——————————————————————————————————————————————
## included variable NULL
## strategy forward
## metric SL & IC(3/2)
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## intercept 1
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## ——————————————————————————————————————————————
## Dependent reports numeric
## Independent card factor
## Independent age numeric
## Independent income numeric
## Independent share numeric
## Independent expenditure numeric
## Independent owner factor
## Independent selfemp factor
## Independent dependents numeric
## Independent months numeric
## Independent majorcards numeric
## Independent active numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under forward with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn SL
## ——————————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 1
## 1 card 2 2 8.58772215018167e-235
## 2 active 3 3 7.53645358152151e-61
## 3 expenditure 4 4 0.000166720608577516
## 4 months 5 5 0.00149658752249566
## 5 owner 6 6 0.000289704544697974
## 6 majorcards 7 7 0.00853660247622726
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Selection Process under forward with IC(3/2)
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn IC(3/2)
## —————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 2998.46727451502
## 1 card 2 2 2161.5438034863
## 2 active 3 3 1970.86790229148
## 3 expenditure 4 4 1962.20751098534
## 4 months 5 5 1954.79845463846
## 5 owner 6 6 1942.91316602911
## 6 majorcards 7 7 1937.16269545213
## 7 income 8 8 1937.02611660997
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 5. Parameter Estimates for reports under forward with SL
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## ———————————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) -0.298643659038298 0.109685399601467 -2.72272937075851 0.00647450714191608
## cardyes -2.70352225795467 0.117195939295856 -23.0683953232351 9.61653770781961e-118
## active 0.0654296707660895 0.00399754905523789 16.367446618412 3.26632926223346e-60
## expenditure 0.000672431213470284 0.000177638845762774 3.7853838251588 0.000153471518736066
## months 0.00212461501050368 0.000530320086460442 4.00628802254911 6.16804294228678e-05
## owneryes -0.343769864333075 0.0926480304376423 -3.71049295607479 0.000206856052610903
## majorcards 0.274039347897116 0.104512901787066 2.62206237901078 0.00873994324369939
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 6. Parameter Estimates for reports under forward with IC(3/2)
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## ———————————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) -0.370064654071278 0.122569709425332 -3.0192178459615 0.00253428230748205
## cardyes -2.69192183928202 0.117368471922898 -22.9356469857629 2.04939317764668e-116
## active 0.064733432332392 0.00402555478259611 16.0806238713375 3.48848117936479e-58
## expenditure 0.000598437222203223 0.000185715652467924 3.22233055884496 0.00127152347189398
## months 0.00202983041537344 0.000534807209066601 3.79544325686279 0.000147379901803214
## owneryes -0.368861130713364 0.0948758378946128 -3.88783001972632 0.000101144415795061
## majorcards 0.257131033787658 0.105294602110372 2.44201534204125 0.014605525916605
## income 0.0328317155222304 0.0253062467170453 1.29737593604176 0.194501868167293
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
- Visulization of the selection process using
forwardstrategy under information criteronIC(3/2)andSL.
plot(res8)## $forward
Example9:
type of regression:
poissonresponse:
reportspredictors: all variables except
reportsvariable selection strategy:
forwardandbidirectionselection metric: run
AICandIC(3/2)parallellyforce
cardandmonthsin all models.
formula = reports ~ .
res9 <- stepwise(formula = formula,
data = CreditCard,
type = "poisson",
include=c("card","months"),
strategy = c("forward","bidirection"),
metric = c("IC(3/2)","AIC")
)
res9## Table 1. Summary of Parameters
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Parameter Value
## ———————————————————————————————————————————————————————
## included variable card months
## strategy forward & bidirection
## metric IC(3/2) & AIC
## tolerance of multicollinearity 1e-07
## multicollinearity variable NULL
## intercept 1
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 2. Type of Variables
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable type Variable name Variable class
## ——————————————————————————————————————————————
## Dependent reports numeric
## Independent card factor
## Independent age numeric
## Independent income numeric
## Independent share numeric
## Independent expenditure numeric
## Independent owner factor
## Independent selfemp factor
## Independent dependents numeric
## Independent months numeric
## Independent majorcards numeric
## Independent active numeric
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 3. Selection Process under forward with IC(3/2)
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn IC(3/2)
## —————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 2998.46727451502
## 0 card months 2 3 2153.23072693751
## 1 active 4 4 1963.56167073775
## 2 owner 5 5 1952.16998343778
## 3 expenditure 6 6 1942.91316602911
## 4 majorcards 7 7 1937.16269545213
## 5 income 8 8 1937.02611660997
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 4. Selection Process under forward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn AIC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 2998.96727451502
## 0 card months 2 3 2154.73072693751
## 1 active 4 4 1965.56167073775
## 2 owner 5 5 1954.66998343778
## 3 expenditure 6 6 1945.91316602911
## 4 majorcards 7 7 1940.66269545213
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 5. Parameter Estimates for reports under forward with IC(3/2)
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## ———————————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) -0.370064654071277 0.122569709425332 -3.01921784596149 0.0025342823074821
## cardyes -2.69192183928202 0.117368471922898 -22.935646985763 2.04939317764585e-116
## months 0.00202983041537344 0.000534807209066601 3.79544325686279 0.000147379901803215
## active 0.064733432332392 0.00402555478259611 16.0806238713375 3.48848117936519e-58
## owneryes -0.368861130713364 0.0948758378946128 -3.88783001972632 0.000101144415795062
## expenditure 0.000598437222203223 0.000185715652467924 3.22233055884497 0.00127152347189396
## majorcards 0.257131033787657 0.105294602110372 2.44201534204124 0.0146055259166053
## income 0.0328317155222305 0.0253062467170453 1.29737593604176 0.194501868167292
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 6. Parameter Estimates for reports under forward with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## ———————————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) -0.298643659038298 0.109685399601467 -2.72272937075851 0.00647450714191609
## cardyes -2.70352225795467 0.117195939295856 -23.0683953232351 9.61653770781724e-118
## months 0.00212461501050368 0.000530320086460441 4.00628802254911 6.16804294228673e-05
## active 0.0654296707660895 0.0039975490552379 16.367446618412 3.26632926223498e-60
## owneryes -0.343769864333074 0.0926480304376423 -3.71049295607478 0.00020685605261091
## expenditure 0.000672431213470282 0.000177638845762774 3.78538382515879 0.000153471518736071
## majorcards 0.274039347897117 0.104512901787067 2.62206237901079 0.0087399432436993
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 7. Selection Process under bidirection with IC(3/2)
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn IC(3/2)
## —————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 2998.46727451496
## 0 card months 2 3 2153.23072693751
## 1 active 4 4 1963.56167073775
## 2 owner 5 5 1952.16998343778
## 3 expenditure 6 6 1942.91316602911
## 4 majorcards 7 7 1937.16269545213
## 5 income 8 8 1937.02611660997
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 8. Selection Process under bidirection with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Step EnteredEffect RemovedEffect NumberEffectIn NumberParmsIn AIC
## —————————————————————————————————————————————————————————————————————————————————————
## 0 1 1 1 2998.96727451496
## 0 card months 2 3 2154.73072693751
## 1 active 4 4 1965.56167073775
## 2 owner 5 5 1954.66998343778
## 3 expenditure 6 6 1945.91316602911
## 4 majorcards 7 7 1940.66269545213
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 9. Parameter Estimates for reports under bidirection with IC(3/2)
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## ———————————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) -0.370064654071274 0.122569709425332 -3.01921784596147 0.00253428230748225
## cardyes -2.69192183928203 0.117368471922898 -22.935646985763 2.04939317764651e-116
## months 0.00202983041537344 0.000534807209066601 3.79544325686279 0.000147379901803215
## active 0.064733432332392 0.00402555478259611 16.0806238713375 3.48848117936579e-58
## owneryes -0.368861130713364 0.0948758378946128 -3.88783001972632 0.000101144415795061
## expenditure 0.000598437222203224 0.000185715652467925 3.22233055884496 0.00127152347189399
## majorcards 0.257131033787657 0.105294602110372 2.44201534204124 0.0146055259166053
## income 0.0328317155222304 0.0253062467170453 1.29737593604176 0.194501868167292
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
##
## Table 10. Parameter Estimates for reports under bidirection with AIC
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
## Variable Estimate Std. Error z value Pr(>|z|)
## ———————————————————————————————————————————————————————————————————————————————————————————————————
## (Intercept) -0.298643659038296 0.109685399601467 -2.72272937075849 0.00647450714191642
## cardyes -2.70352225795467 0.117195939295856 -23.0683953232351 9.61653770782276e-118
## months 0.00212461501050368 0.000530320086460442 4.00628802254912 6.16804294228671e-05
## active 0.0654296707660893 0.00399754905523789 16.367446618412 3.26632926223632e-60
## owneryes -0.343769864333074 0.0926480304376423 -3.71049295607479 0.000206856052610908
## expenditure 0.000672431213470281 0.000177638845762775 3.78538382515877 0.00015347151873608
## majorcards 0.274039347897117 0.104512901787067 2.62206237901079 0.00873994324369927
## ‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
- Visulization of the selection process using
bidirectionstrategy under information criteronIC(3/2)andAIC.
plot(res9)## $forward
##
## $bidirection
## $bidirection
6 Session info
## R version 4.3.2 (2023-10-31)
## Platform: aarch64-apple-darwin20 (64-bit)
## Running under: macOS Sonoma 14.2.1
##
## Matrix products: default
## BLAS: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.11.0
##
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## time zone: America/New_York
## tzcode source: internal
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
<<<<<<< HEAD
## [1] kableExtra_1.3.4 knitr_1.45 StepReg_1.5.0 BiocStyle_2.22.0
##
## loaded via a namespace (and not attached):
## [1] httr_1.4.5 sass_0.4.8 viridisLite_0.4.2
## [4] jsonlite_1.8.8 splines_4.1.3 bslib_0.6.1
## [7] shiny_1.8.0 askpass_1.2.0 BiocManager_1.30.20
## [10] highr_0.10 xlsxjars_0.6.1 fontLiberation_0.1.0
## [13] yaml_2.3.8 ggrepel_0.9.5 gdtools_0.3.5
## [16] pillar_1.9.0 lattice_0.21-8 glue_1.7.0
## [19] uuid_1.2-0 digest_0.6.34 promises_1.2.1
## [22] rvest_1.0.3 colorspace_2.1-0 htmltools_0.5.7
## [25] httpuv_1.6.14 Matrix_1.5-3 gfonts_0.2.0
## [28] fontBitstreamVera_0.1.1 pkgconfig_2.0.3 httpcode_0.3.0
## [31] bookdown_0.33 purrr_1.0.2 xtable_1.8-4
## [34] webshot_0.5.4 scales_1.3.0 svglite_2.1.1
## [37] later_1.3.2 officer_0.6.4 fontquiver_0.2.1
## [40] tibble_3.2.1 openssl_2.1.1 generics_0.1.3
## [43] farver_2.1.1 ggplot2_3.4.4 ellipsis_0.3.2
## [46] cachem_1.0.8 withr_3.0.0 cli_3.6.2
## [49] survival_3.5-5 magrittr_2.0.3 crayon_1.5.2
## [52] mime_0.12 evaluate_0.23 fansi_1.0.6
## [55] xml2_1.3.3 textshaping_0.3.7 tools_4.1.3
## [58] data.table_1.15.0 lifecycle_1.0.4 stringr_1.5.1
## [61] xlsx_0.6.5 flextable_0.9.4 munsell_0.5.0
## [64] zip_2.3.1 compiler_4.1.3 jquerylib_0.1.4
## [67] systemfonts_1.0.5 rlang_1.1.3 grid_4.1.3
## [70] rstudioapi_0.14 labeling_0.4.3 rmarkdown_2.25
## [73] gtable_0.3.4 curl_5.2.0 R6_2.5.1
## [76] dplyr_1.1.4 fastmap_1.1.1 utf8_1.2.4
## [79] ragg_1.2.7 rJava_1.0-6 stringi_1.8.3
## [82] crul_1.4.0 Rcpp_1.0.12 vctrs_0.6.5
## [85] tidyselect_1.2.0 xfun_0.42